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Abstract 

In recent years there has been a renewed interest in understanding the levels and 
trends in high school graduation in the U.S. A big and influential literature has 
argued that the “true” high school graduation rate remains at an unsatisfactory 
level, and that the graduation rates for minorities (Blacks and Hispanics) are 
alarmingly low. In this paper we take a closer look at the different measures of high 
school graduation which have recently been proposed and which yield such low 
estimates of graduation rates. We argue that the nature of the variables in the 
Common Core of Data, the dataset maintained by the U.S. Department of 
Education that is the main source for all of the new measures, requires caution in 
calculating graduation rates, and the adjustments that have been proposed often 
impart significant downward bias to the estimates. 
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El uso de datos administrativos para estimar tasas de graduacion: Desaflos, 
soluciones propuestas y sus peligros 

Resumen 

En los ultimos anos ha habido un renovado interes en la comprension de los 
niveles y tendencias de la graduacion de la escuela secundaria en los EE.UU. Una 
gran e influyente literatura ha argumentado que la "verdadera" tasa de graduacion 
escolar se mantiene en un nivel insatisfactorio, y que las tasas de graduacion de las 
minorlas (negros e hispanos) son alarmantemente bajas. En este trabajo se examina 
con mayor atencion los indicadores de graduacion de escuelas secundarias que 
recientemente se han propuesto, que dan por resultado esas estimaciones de las 
tasas de graduacion excesivamente bajas. Nosotros sostenemos que la naturaleza de 
las variables en la Base Central de Datos (Common Core of Data), desarrollada por 
Departamento de Educacion de los EE.UU. que es la principal fuente para todas 
las nuevas medidas, demandan cautela en el calculo de las tasas de graduacion, ya 
que las adaptaciones que se han propuesto, ha menudo implican un importante 
sesgo para reducir las estimaciones de graduacion. 

Palabras clave: graduacion de la escuela secundaria; de medicion; base comun de 
datos. 


High school graduation remains one of the most significant and basic indicators of 
educational attainment. In a world where more educated workers earn significantly higher wages, it 
is also an important indicator of future earnings and other labor market outcomes. In recent years 
there has been a renewed interest in understanding the levels and trends in high school graduation in 
the U.S. In particular, a big and influential literature has argued that the tme high school graduation 
rate in the U.S. remains at an unsatisfactory level and that the graduation rates for minorities (Blacks 
and Hispanics) are alarmingly low. These studies include Greene (2001), Greene and Forster (2003), 
Greene and Winters (2005, 2006), Swanson (2003, 2004), and Education Week (2006, 2008) — see 
Appendix Table A-l, which summarizes these estimates. In earlier work (Mishel and Roy, 2006), we 
had examined data from diverse sources — including school and district administrative data from the 
U.S. Department of Education, longitudinal surveys which follow individual students over time, 
household surveys and the decennial census — and found this claim (particularly, the assertion that 
Blacks and Hispanics have only a 50% chance of graduating from high school with a regular 
diploma) to be seriously inaccurate. In this paper we take a closer look at the different measures of 
high school graduation which have recently been proposed and which yield such low estimates of 
graduation rates. We argue that researchers and policymakers must remain cautious in using 
variables in the Common Core of Data (CCD), the dataset maintained by the U.S. Department of 
Education that is the main source for all of the new measures; 1 the adjustments that have been 
proposed to adjust for its flaws often impart significant downward bias to the estimates. 

The rest of the paper is broadly divided into two parts. In the next section, we analyze the 
nature of the variables recorded in the Common Core of Data (CCD), emphasizing the particular 


M The CCD is a statistical database maintained by the National Center for Education Statistics 
(NCES), an arm of the U.S. Department of Education. 
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features that make it imperative for researchers to be careful when calculating graduation rates. This 
is more so when one wants to compare graduation rates across states or over time. In the following 
section, we critically examine the most influential and popular graduation rate measures that have 
been proposed recently. We argue that when the new studies adjust CCD data to account for size of 
the entering ninth grade class and population growth during high school years, the new measures 
still impart significant downward bias to the estimates; the bias is much worse for Black and 
Hispanic graduation rates. We also briefly discuss two recent studies, Warren and Halpern-Manners 
(2007) and Heckman and LaFontaine (2007), which address discrepancies in estimated graduation 
rates from different sources. Two appendices provide supplements to this analysis. Appendix A lists 
estimates of graduation rates from recent studies which mostly or wholly rely on CCD data. 
Appendix B discusses whether the population adjustments used in Greene and Winters (2006), to 
account for differential migration or population growth across states and school districts, are 
reasonable and valid. 


Challenges of CCD Data 

The enrollment and diploma counts in the Common Core of Data are the only data available 
at the state and local (school and school district) levels, so it is not surprising that researchers have 
tried to compute graduation rates with these data. We believe there are important limitations to any 
computation of graduation rates using the CCD, setting aside any question of the quality and 
completeness of the data. This is because the CCD does not measure high school graduation rates of 
entering ninth graders. Consequently, researchers must estimate graduation rates by constructing 
what they describe as cohort graduation rates based on enrollment and diploma data for particular 
years. (These measures are different from longitudinal rates that would be calculated from tracking 
individual students through school.) 

CCD's Limits 

There are several data limitations that frustrate this effort, including an inability to 
distinguish between on-time diplomas and late diplomas, difficulty in approximating the true size of 
entering ninth grade cohorts, difficulty in estimating transfers in and out of school districts, and 
number and types of exit options including definition of regular diplomas which differ from state to 
state, making a straightforward comparison problematic and possibly misleading. We discuss these 
in more detail below. 

Diploma counts and cohorts. It is not generally understood that the diploma counts in the 
CCD include all diplomas, on-time or not, even though some people refer to the rates calculated 
using the CCD as on-time rates. Unless very specific assumptions are made about the distribution of 
diplomas among on-time graduates and late graduates, whose veracity has to be checked by data 
from independent sources, it is not possible to compute 4-year or 5-year or even 6-year graduation 
rates using diploma data from the CCD. This is particularly problematic as there are trends in high 
school graduation rates, and static assumptions using diploma data from a particular year and a 
different survey are likely to be incomplete. 

Entering ninth graders. Researchers generally acknowledge that the graduation rate should 
reflect how many entering ninth graders complete high school with a diploma. Unfortunately, the 
CCD does not report entering ninth graders', rather, it reports ninth grade enrollment, including 
students who are repeating 9th grade (that is, who entered 9th grade the prior year or even earlier). 
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This is an important distinction because there is substantial retention of students, particularly 
minorities, in 9th grade and sometimes in 10th grade. We find that for the nation as a whole, there 
are 12—13% more students in 9th grade in public schools than in the 8th grade in the previous year; 
for Blacks and Hispanics the rate is more like 25%. Since retention is larger for some demographic 
groups, and in some states compared to others, the method for accounting for retention — or not 
doing so — can greatly affect racial comparisons and state comparisons. 

This can be clearly seen in Table 1, which shows 9th grade enrollment in 2003-04 as a 
percentage of the previous year’s (2002-03) 8th grade enrollment, disaggregated by race. For almost 
all state-race pairs, 9th grade enrollment in 2003-04 is well above the previous year’s 8th grade 
enrollment. The underlying reason is grade retention at the 9th grade, which particularly affects 
minorities. 2 For the nation as a whole, there were 22% more Blacks and 23% more Flispanics in 9th 
grade in 2003-04 compared to the corresponding 8th grade enrollment in the previous year. 3 The 
percentages vary widely across states — e.g., while in states like Mississippi and Utah the bulge is 
smaller and similar across racial groups, in Nebraska, New York, and Wisconsin, 9th grade 
enrollments for both Blacks and Flispanics are more than 30% higher than the previous year’s 8th 
grade enrollment. 

Table 1 

Ninth-grade enrollment in public schools in 2003—04 compared to 8th grade enrollment in public 
schools in 2002-03, by state (%) 


State 

White 

Black 

Hispanic 

Asian 

Alaska 

107 

100 

106 

103 

Alabama 

106 

113 

120 

118 

Arkansas 

102 

104 

112 

120 

Arizona 

122 

127 

121 

128 

California 

104 

114 

117 

109 

Colorado 

106 

122 

119 

108 

Connecticut 

103 

121 

127 

110 

Delaware 

110 

111 

111 

111 

Florida 

120 

137 

129 

122 

Georgia 

111 

123 

131 

111 

Hawaii 

117 

115 

116 

119 

Iowa 

107 

123 

124 

108 


2 Two other possible explanations have often been advanced for this heaping of students at the 9th 
grade (the 9th grade “bulge”). One is population growth, as migration of students from one state to another 
between the 8th and the 9th grades can increase the populations in some states. Flowever, this is unlikely to 
explain the bulge at the national level, where the only increase can come through net international migration. 
While net immigration is an important factor in overall population growth, most of it is concentrated among 
the Flispanic population, and recent immigrants are much less likely to enroll in schools and thereby be 
included in enrollment counts. The second explanation is the transfer of students from private to public 
schools between the 8th and 9th grades. It is true that private schools educate a significantly lower percentage 
of the population in the high school grades compared to the middle school grades. It is also true that the 
importance and spread of private schools is different in different states, which may account for some of the 
difference in 9th grade bulge across states. However, as the analysis below shows, the private-to-public school 
transfer can only explain a small part of the bulge. Most of the bulge is concentrated among the Blacks and 
the Hispanics, for whom the issue of transfer from private to public schools is much less important than it is 
for White students. 

3 This table excludes Washington, D.C., as enrollment data for 2003—04 were not available. 
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State 

White 

Black 

Hispanic 

Asian 

Idaho 

105 

98 

108 

116 

Illinois 

106 

123 

117 

109 

Indiana 

105 

117 

118 

109 

Kansas 

104 

109 

127 

106 

Kentucky 

112 

119 

132 

105 

Louisiana 

107 

85 

109 

104 

Massachusetts 

103 

125 

124 

108 

Maryland 

108 

125 

131 

111 

Maine 

98 

112 

106 

98 

Michigan 

109 

132 

119 

108 

Minnesota 

103 

110 

114 

102 

Missouri 

106 

114 

116 

112 

Mississippi 

104 

107 

108 

106 

Montana 

105 

116 

107 

99 

North Carolina 

110 

123 

134 

113 

North Dakota 

103 

99 

125 

87 

Nebraska 

106 

144 

131 

112 

New Hampshire 

105 

114 

115 

103 

New Jersey 

101 

117 

121 

105 

New Mexico 

108 

112 

118 

117 

Nevada 

112 

123 

130 

125 

New York 

105 

139 

142 

131 

Ohio 

108 

127 

124 

113 

Oklahoma 

104 

115 

118 

105 

Oregon 

103 

106 

114 

106 

Pennsylvania 

106 

123 

128 

119 

Rhode Island 

107 

122 

124 

112 

South Carolina 

115 

127 

144 

124 

South Dakota 

102 

100 

115 

104 

Tennessee 

110 

117 

136 

116 

Texas 

110 

125 

128 

112 

Utah 

101 

101 

105 

104 

Virginia 

110 

124 

129 

115 

Vermont 

103 

105 

121 

100 

Washington 

109 

117 

122 

112 

Wisconsin 

111 

134 

135 

109 

West Virginia 

108 

114 

120 

110 

Wyoming 

103 

103 

116 

105 

United States 

108 

122 

123 

113 


This table excludes Washington DC, as enrollment data for 2003—04 were not available. Source: 
Common Core of Data, National Center for Education Statistics. 


In the presence of grade retention, and in particular when the extent of grade retention 
differs significantly across states, graduation rates using CCD enrollment numbers, which fail to 
distinguish between entering ninth-graders and repeating ninth-graders will be biased. For example, 
states with a stricter retention policy will appear to push many of their students out of school even 
when their true graduation rates might be much higher. 
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Moreover, the trends in grade retention can often change from year to year, affecting 
comparisons not only across states but across years as well. Figure 1 is taken from Mishel and Roy 
(2006) and shows that since 1988 there has been a steady increase in the overall size of the 9th grade 
as compared to previous year’s 8th grade. The trends are different for different racial groups — with 
the white rate slightly inching up, while the Black and Hispanic rates slightly decline in recent years 
after reaching a plateau of 25%. 

130 




100 

1988 1990 1992 1994 1996 1998 2000 2002 2004 

Figure 1 . Patio of 9 th grade enrollment to previous year' s 8th grade enrollment, 1988—2004 
Source: Authors' calculations from CCD database. 100 = a one-to-one ratio. Greater than 100 
indicates more 9th than 8th graders. The figures for individual races pertain only to the 40 states 
for which we have continuous data over this period, and are available only from 1992-93 
onwards. The figure for Total includes all 50 states. 

Transfers. The diploma counts reported in the CCD include diplomas that are earned by 
students who transferred into a school, district, or state. Consequently, graduation rates can be 
distorted in areas where there are substantial increases or decreases in the student population. Since 
the CCD only contains data on total enrollment by grade (and breakdown by race/ ethnicity and 
gender), it is not possible to separate transfers from other enrollment changes (e.g., out-transfers 
distinguished from those who have dropped out). Some computations using the CCD do not 
account for this, while others do (e.g., Warren, 2005). The results of adjusting for transfers in and 
transfers out are problematic; however, making no adjustment may be equally problematic, 
particularly for some urban school districts where student mobility rates are extremely high. 

Number and Types of Exit Options. Each state defines what it means to complete high 
school including graduating with a diploma, and that definition can change over time, frustrating the 
need to have as consistent a definition as possible. As Guy et al. (1999) show, the number and types 
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of exit options available in each state differ significantly from state to state. This heterogeneity is 
particularly true for special education students, who may or may not have exit options based on 
occupational diplomas, IEPs, or diplomas based on attendance, in addition to a standard diploma. 

This heterogeneity is partly reflected in the fact that several states have different categories 
of completion as reported in the CCD. The CCD groups all completers in three categories — diploma 
recipients , high school equivalency recipients , and other high school completers . 4 The first category 
(diploma recipients) form the data used by all CCD-based studies of high school graduation rates, 
including those by Greene (2001), Greene and Forster (2003), Greene and Winters (2005, 2006), 
Swanson (2003, 2004), Warren (2005) and by the National Center for Education Statistics itself 
(Seastrom, Hoffman, Chapman, & Stillwell, 2005, 2007). The second category is supposed to contain 
GEDs and similar equivalency documents, but Department of Education officials believe these data 
are not very reliable. The third category is supposed to contain those with certificates of completion 
or attendance. The cross-state heterogeneity is significant — in states like Georgia, Oregon, and 
Alabama, the share of the third category is more than 9% of all completers, while in states like 
California, Illinois, and Massachusetts, there are no completers in this category. In the absence of 
information about the nature of these completion options, and how well they approximate a regular 
high school diploma, estimates of graduation rates in these states might be biased and are not 
reliable bases for measures to be compared with other states. 

Reliability of CCD counts on enrollment. Greene and Winters (2006) argue that enrollment 
and diploma counts as reported in CCD are quite reliable: 

CCD establishes standards and procedures for states to collect and report 
enrollment and diploma data. If states do not meet those standards or follow 

those procedures, their data are not reported It should not be difficult for 

states to track enrollment and diplomas. Enrollment counts are based on schools 
taking attendance, which schools are very good at doing. One reason schools are 
likely to keep accurate attendance is that enrollment counts are the basis for 
school funding by state and federal governments. Further, because attendance 
determines how much money state and federal governments allot to schools, 
these higher levels of government are inclined to check and ensure the accuracy 
of attendance figures. 

However, though the NCES strives for an accurate and uniform count of enrollments and 
diplomas, and the CCD data are believed to be generally reliable, there has not been any 
independent estimate of veracity of these data. Dorn (2006) has highlighted problems with CCD 
enrollment counts in Detroit, and it is likely that such problem persists in many other schools 
and districts too. This is particularly important as CCD data are used to estimate graduation 
rates not only for big cities and states, but also for smaller school districts — it is not uncommon 
to find in CCD data significant jumps in enrollment and / or diplomas from year to year. 

Table 2 and Table 3 show some suggestive evidence about the instability of graduation rates 
calculated using CCD enrollment and diploma counts. Table 2 calculates the graduation rates for 


4 The definitions of these are as follows, obtained from the website of the Common Core of Data 
(CCD), NCES, U.S. Department of Education. Total Diploma Recipients — This is the total number of 
students in a state who received a diploma during the previous school year and subsequent summer school. 
Total HS Tquivalency Recipients — This is the number of students in a state ages 19 or younger who 
received a formal document certifying that an individual met the state requirements for high school 
graduation equivalency. Total Other HS Completers — This is the number of students in a state who received 
a certificate of attendance or other certificate of completion, in lieu of a diploma during the previous school 
year and subsequent summer. 
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Detroit City School District using the Swanson CPI index, following Dorn (2006). As is evident, 
there was a dramatic decline in graduation rate not only from 2001-02 to 2002-03 (from 74% to 
22%), which could be ignored as faulty data, but also from 2003-04 to 2004—05, when the 
graduation rate jumped from 25% to 38%. 

Table 2 


CPI Graduation Rates in Detroit City School District 
(includes all public schools in Detroit except charter schools) 


Measure 

2001-02 

2002-03 

2003-04 

2004-05 

2005-06 

8th grade enrollment 

9,975 

12,048 

12,357 

11,860 

10,513 

9th grade enrollment 

14,494 

20,025 

17,837 

16,832 

15,690 

10th grade enrollment 

9,291 

11,275 

9,899 

9,326 

9,820 

11th grade enrollment 

6,355 

7,795 

7,421 

6,581 

7,365 

12th grade enrollment 

4,618 

6,020 

5,244 

5,604 

5,352 

Diplomas issued 

5,540 

5,975 

4,975 

5,673 


Swanson CPI 

74.2% 

21.7% 

24.9% 

37.9% 



Source: Authors’ calculations using data from CCD. 


Table 3 shows that there is a lot of similar year-to-year instability even for graduation rates 
calculated at the state level. Here we use estimates of graduation rates reported in Haney et al (2004), 
who use the simple 8th-grade-to-graduation rate measure (termed the Basic Completion Rate or 
BCR-8 by Warren (2005)) — the table shows states where graduation rates jumped by 5% or more 
across consecutive years.’ It is important to note that these fluctuations are not due to exogenous 
adjustments or assumptions imposed by researchers, but rather due to the data reported in CCD 
itself. If the CCD were indeed a most reliable count of enrollments and diplomas, then it would be 
unlikely to see this much instability in the data, particularly when they are aggregated at the state 
level. Some of the jumps have occurred in big states like Ohio and Texas. (The district level 
graduation rates would show even more volatility.) 

Phelps (2005) discusses in detail the issue of enrollment counts as reported by school 
districts and stored in the CCD. He makes the distinction between student membership, enrollments 
and attendance, and argues that researchers wishing to construct valid graduation rates using data 
from the CCD should be aware of the subtle differences among these categories. He also notes how 
migration of students is not consistent across different states and school districts and has the 
potential to impart significant bias to measured graduation rates. 


5 BCR-8 is simply the number of diploma recipients in the spring of year t+5 divided by enrollment 
in 8th grade in the fall of year /. For example, the graduation rate for the Class of 1996 (1995—96) is the 
number of diplomas earned by summer of 1996 divided by 8th grade enrollment in 1991—92. 
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Table 3 

Instability in CCD graduation rates across states: Increases or declines in graduation rates by at 
least 5% across consecutive years 


State 

First year 

Graduation Rate 
Year 1 Year 2 Year 3 

Arizona 

1989-1990 

76% 

82% 


Arizona 

1992-1993 

81% 

75% 

68% 

Connecticut 

1998-1999 

79% 

85% 


Hawaii 

1989-1990 

96% 

85% 


Kentucky 

1993-1994 

83% 

78% 

75% 

Louisiana 

1991-1992 

59% 

64% 


Massachusetts 

1988-1989 

79% 

85% 


Minnesota 

1996-1997 

81% 

87% 


Mississippi 

1988-1989 

60% 

66% 


Nevada 

1988-1989 

73% 

79% 


Nevada 

1995-1996 

68% 

76% 


New Jersey 

1996-1997 

90% 

82% 


New Jersey 

1998-1999 

83% 

91% 


New Mexico 

1996-1997 

72% 

66% 


Ohio 

1994-1995 

83% 

78% 


Oregon 

1994-1995 

76% 

70% 


South Carolina 

1988-1989 

70% 

64% 


South Dakota 

1998-1999 

75% 

80% 


Tennessee 

1995-1996 

71% 

66% 

62% 

Texas 

1990-1991 

74% 

69% 


Vermont 

1988-1989 

76% 

85% 

77% 

Wyoming 

1999-2000 

78% 

73% 



Source: Authors’ calculations from data reported in Haney et al. (2004), Table 4. The graduation rates 
reported are simple eighth-grade-to-diploma rates (termed the Basic Completion Rate or BCR-8 by 
Warren, 2005). 

Diploma definitions. Not only does each state have different numbers and types of exit 
options, every state has its own definition for a regular diploma (see the NCES report of the Task 
Force on Graduation, Completion, and Dropout Indicators, 2005, and the report Diploma Counts', 
Education Week, 2006). According to data collected by the Education Commission of the States and 
reported by Education Week, state requirements for obtaining a standard diploma for the 2005-06 
school year range from a low of 1 3 total credits in California, Wisconsin, and Wyoming to a high of 
24 total credits in Alabama, Florida, South Carolina, and West Virginia (Lloyd, 2006). In nine states, 
students who want a standard diploma has to earn 23-24 credits, while six states only require 13-16 
credits. A few other states, including Massachusetts, leave the number of required credits as an 
option for local school boards. 

Moreover, states often change the requirements for diplomas, as New York did recently: 
Requirements for earning a local diploma went up from 20.5 credits to 22 credits. None of these 
changes will be reflected in the CCD data that the recent studies — particularly those by Swanson and 
Greene and his coauthors — use for comparing graduation rates across states and over time. Without 
the additional adjustments that none of the studies referred to above makes, one cannot conduct 
either a state-by-state or a year-by-year comparison with the existing CCD data, particularly if the 
goal is to judge student or school performance. 
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The pitfalls in comparing across states using CCD-based graduation rates can be illustrated 
using Table 4. We show two pairs of states — Arkansas and Georgia, and North and South Carolina. 
Arkansas and Georgia are both southern states, as are North and South Carolina. However, the 
graduation rates calculated by Swanson and Greene, show large differences in graduation states 
between these pairs of states. For example, while Arkansas has an overall graduation rate of 72%, 
Georgia’s rate is only 56% — a difference of 16% (Swanson, 2003, 2004). The Greene method yields 
a similarly large difference — 74% graduation rate in Arkansas compared to 56% in Georgia (Greene 
and Winters, 2006). These differences persist across racial groups — the Swanson CPI shows a 18% 
gap in graduation rates for Blacks (64% versus 46%), while the Greene method yields an even larger 
21% gap (69% versus 48%). The picture is basically the same if we compare North and South 
Carolinas — the former has a graduation rate of 66% (Swanson CPI) compared to 53% for the latter. 
The difference is even larger for the Greene method, 69% against 54%. 

However, the difference between these pairs of states is minimal when it comes to 4th and 
8th grade reading and math performance in the National Assessment of Educational Progress 
(NAEP) data. As Table 4 shows, the average scale scores of students in Georgia and Arkansas are 
very similar in these national tests, despite the supposedly higher attainment in Arkansas — for Black 
students, Georgia performs better than Arkansas. (Hispanic students in Arkansas perform better 
than those in Georgia, but Hispanic students form a negligible portion of the student population in 
Arkansas.) It is difficult to reconcile a 18-21% gap in graduation rates for Blacks between Arkansas 
and Georgia when Blacks in Georgia score significantly higher than their counterparts in Arkansas in 
reading and mathematics at both the 4th and 8th grades. The simplest explanation is that Greene's 
and Swanson's measures are unreliable indicators of the relative performance of Black students in 
Arkansas vis-a-vis Georgia. 6 7 In fact, most of the difference can be explained by the difference in the 
types of exit options available in either state — e.g., how Georgia's CCD report categorizes 9% of all 
completers in the “Other High School Completers” category (i.e., these students are not included in 
the regular diploma counts). 

Outside the South, while students in Massachusetts score at the top nationally — for example, 
in 2007 the NAEP 8th grade scores (average scale score) for Massachusetts is 298 in mathematics 
and 273 in reading, both being the highest in the nation — in terms of graduation the state is ranked 
far below by recent studies. Greene and Winters rank it 28th in the nation (Class of 2003) and 21st 
in the nation (Class of 2002), while Warren ranks it 17th (Class of 2002) and Swanson (2004) ranks it 
26th (Class of 2001). 


6 Graduation rate measures might still be meaningful if properly calculated and under certain 
circumstances for within-state comparisons; however, as things currently stand, they are not of much use if 
we want to compare graduation rates across states. 

7 There does not seem to be a noticeable difference in the number of credits required in each state to 
earn a standard diploma (Lloyd, 2006). 
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Table 4 

NAEP performance of neighboring Southern states with different graduation rates as calculated by 
Greene and Swanson 


Arkansas 

Georgia 

NC 

SC 

Racial Composition (% of enrollment, Class of 2003) 



Asian 

0.8 

2.5 

2.0 

1.1 

Hispanic 

2.2 

6.2 

6.0 

2.7 

Black 

23.0 

38.2 

31.3 

41.5 

White 

73.5 

53.0 

59.2 

54.5 

Swanson graduation rates, CPI, Class of 2003 



All 

71.8 

56.3 

66.2 

52.5 

Asian 

— 

75.3 

77.6 

— 

Hispanic 

— 

39.5 

52.9 

— 

Black 

64.3 

45.9 

57.7 

— 

White 

74.8 

63.1 

71.3 

— 

Greene graduation rates, Class of 2003 




All 

74 

56 

69 

54 

Asian 

— 

— 

— 

— 

Hispanic 

— 

— 

— 

— 

Black 

69 

48 

62 

— 

White 

77 

64 

76 

— 

NAEP Performance in 2005- 

-All 




4th grade reading 

217 

214 

217 

213 

4th grade mathematics 

236 

234 

241 

238 

8 th grade reading 

258 

257 

258 

257 

8th grade mathematics 

272 

272 

282 

281 

NAEP Performance in 2005- 

— Hispanic 




4th grade reading 

212 

203 

204 

215 

4th grade mathematics 

229 

229 

234 

236 

8 th grade reading 

250 

247 

248 

— 

8th grade mathematics 

266 

258 

265 

269 

NAEP Performance in 2005- 

— Black 




4th grade reading 

194 

199 

200 

197 

4th grade mathematics 

214 

221 

225 

223 

8 th grade reading 

236 

241 

240 

242 

8th grade mathemadcs 

243 

255 

263 

263 

NAEP Performance in 2005- 

—White 




4th grade reading 

225 

226 

227 

225 

4th grade mathematics 

242 

243 

250 

250 

8 th grade reading 

266 

268 

267 

267 

8th grade mathematics 

281 

284 

292 

294 


Cells with — had insufficient data for reliable estimates. Source: The NAEP numbers are obtained from 
the website of The Nation’s Report Card, National Center for Education Statistics 
( http:/ / nces.ed.gov/ nationsreportcard /) . The Greene graduation rates are from Greene and Winters 
(2006), while the Swanson graduation rates (CPI) are from Education Week (2006). 
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The CCD collects data on every public school and public school district in the country. 
However, this fact alone does not make it particularly suitable for calculating graduation rates. The 
problem is that the CCD does not track individual students over time, as explained above. The best 
one could do to calculate graduation rates using the CCD data is to compare the number of 
diplomas in a particular year, such as 2006, over the number of entering 9th graders in the fall of 
2002. This is problematic in part because we cannot track students who drop out or join between 
the 9th grade and graduation — that is, we cannot account for leavers and joiners. This is particularly 
important if we are to calculate graduation rates at the state and school district levels, as low-income 
minority youths — whose graduation rates are of the greatest concern — also have the highest rates of 
mobility. Beyond the issue of mobility, the CCD data only has grade-specific enrollments, which 
does not allow a researcher to know the number of entering 9th graders. Estimating the number of 
first-time 9th graders based on 9th grade enrollments — or even a smoothed average of 8th, 9th, and 
10th grade enrollments as Greene and Winters have done — is problematic because of significant 
grade retentions in the 9th grade. The major point here is that having a larger sample, or “census,” 
on enrollment and diploma counts does not necessarily provide accurate graduation rates because 
the CCD is not designed to do so. 

In earlier work, we argue that because of these limitations of the CCD data, graduation rates 
constructed using CCD data should be benchmarked against those obtained from more reliable 
sources (Mishel and Roy, 2006). Longitudinal studies such as the National Educational Longitudinal 
Study (NELS) conducted by the U.S. Department of Education and the National Longitudinal 
Surveys (NLSY) conducted by the U.S. Department of Labor do not suffer from either of these 
problems. For example, the NELS began with students in their 8th grade, and then followed them 
over the next 12 years. This data set gives us the correct rates of high school graduation, including 
rates of on-time completion and completion via alternative methods like the GED. (The data set 
also allows us to link the problem of non-completion to the respective families’ socioeconomic 
status and other family and school indicators.) The issue of grade retention or the transferring of 
schools does not affect the NELS results. This is the same for other national longitudinal surveys 
such as the NLSY. Because these surveys have samples that allow them to minimize sampling error 
and measure what is desired, graduation rates using the CCD should at least be benchmarked to 
these longitudinal surveys. 

A true student census would require a national student identifier system, so that we could 
track every student from his or her entering of 9th grade until he or she graduates or drops out. The 
NELS:88 is the big, representative sample version of this idea — for its sample, it does exactly what 
we would do for the universe if it were possible. In statistical terms, saying the CCD is preferred 
over the NELS because it is a “census” is to overvalue reducing sampling error while ignoring much 
worse non-sampling errors. 

Finally, labeling the CCD as a census overlooks the fact that there is only slight quality 
control and checking of the data provided by school districts to their states and by the states to 
NCES. Whether the questionnaire is completed as NCES expects and is done so consistently across 
districts and states and over time is not known because there are no audits done of school district 
respondents. In contrast, the national longitudinal study data are very carefully compiled. 

In general, the CCD data do not provide the measure that we seek: the graduation rate of 
entering 9th graders, either on time or eventual/ final. These problems do not invalidate the use of 
the CCD, but acknowledging them is important. The CCD is certainly not a data set that can be 
described as a longitudinal record of students. At best, each year of the CCD contains a census of 
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enrollment and diplomas, but these are only some of the ingredients in a graduation formula that 
reflects many choices to address the limitations of the CCD. 

Other Issues 

There are some definitional differences between the administrative and the other data, but 
these differences do not explain the large gaps between various estimated graduation rates with 
regular diplomas. For instance, the household-based and longitudinal data include both private and 
public schools, while the CCD data is for public schools alone. Since private schools only comprise 
about 10% of enrollment, even a 20% private-school advantage in graduation would create an 
upward bias in longitudinal and household completion rates of 2%. Furthermore, any such bias 
would affect the completion rates primarily of white students. 8 The longitudinal and household- 
based data also reflect educational attainment eight to thirteen years after what would be a four-year 
on-time completion year. In contrast, the CCD probably reflects the receipt of regular diplomas of 
students who have been enrolled in school that same year. Thus, one difference between the two 
types of data is that the CCD probably doesn’t capture high school completion past the ages of 18 
or 19. Using the NLSY79 data as a guide, we found that later completion among blacks and 
Hispanics boosts graduation by 3% and among whites about 1% (Mishel and Roy, 2006). Again, this 
still leaves a nontrivial gap between the CCD-based measures and all of the other sources of data. 

It is difficult to assess what can be causing these gaps, because there is very little 
documentation and assessment of the CCD data that we could locate, especially since the measures 
are not necessarily consistent across states. This lack of information is a prudent researcher or 
policymaker must be cautious about conclusions when discussing the characteristics of the CCD. 
This lack of information about the CCD has also left us puzzled why analysts give such great 
confidence in their calculations using the CCD data. 9 

Recently-Proposed and Popular Measures of Graduation Rates 

Swanson's Cumulative Promotion Index (CPI) 

One of the most popular new measures of graduation rates has been proposed by Swanson 
(2003, 2004). This synthetic measure compares the enrollment of students in grade »+ / in year /+ / 
to the enrollment of students in grade n in year t (and diplomas in spring of year /+ / to 12th graders 
in fall of year i) and then calculates CPI as the product of these grade-to-grade (or 12th-to-diploma) 
ratios. He calls the measure the Cumulative Promotion Index (CPI), which has been used by many 
organizations, including the Education Week in its annual Diploma Counts issues. Unlike other 
methods, Swanson’s CPI purports to calculate on-time graduation rates. However, the CPI does not 
adjust for grade retention. As a result, Swanson's CPI depends on the size of the 9th grade bulge. 


8 As discussed elsewhere, minorities are much less likely to be in private schools than whites. While 
14% of Whites attend private schools at the elementary level, the figures for Blacks and Hispanics are both 
about 5%. The respective figures at the high school level are 10% for Whites, 3% for Blacks and 4% for 
Hispanics. 

9 Kaufman, Kwon, Klein, & Chapman (2000) discusses the accuracy and comparability of estimates 
from the CCD and the CPS. See also Kaufman’s chapter in Orfield (2004) for more information about the 
CCD. 
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This can be seen in Table 5, where we calculate the CPI both using the 9th grade and the 8th grade. 
That is, we calculate the usual Swanson CPI and also extend it down to the 8th grade by multiplying 
it with the 8th-grade-to-9th-grade progression ratio. 10 

Table 5 


Swanson CPI , 2003, based on 9th grade and extended to 8th grade 



48-state CPI 

Published 50-state 


9th grade 

start 8th grade start 

CPI (9 th grade) 

Group 

(1) 

(2) 

( 3 ) 

Whites 

77.6 

83.4 

76.2 

Blacks 

53.8 

65.3 

51.6 

Hispanics 

58.9 

72.5 

55.6 

Asians/Pacific Islanders 

86.2 

97.1 

77.0 


Source: Swanson’s (published) figures in column (3) are taken from Education Week (2006). These are 
slightly different from the numbers in column (1) calculated by the authors due to data availability for 
jurisdictions. 11 

As is evident by comparing columns (1) and (2), extending the Swanson CPI to the 8th grade 
results in a significant increase in the graduation rates. The increases are particularly large for the 
minorities, going from 54% to 65% for Blacks and from 59% to 73% for Hispanics. This 
contradictory result highlights one important problem with the Swanson CPI. Note that if it were 
indeed an accurate measure of on-time graduation rates, the 8th grade CPI would have to be smaller 
than the 9th grade CPI. 12 The fact that the 8th grade CPI in Table 5 is consistently higher than the 
9th grade CPI — theoretically an impossible result — highlights an important practical problem with 
the Swanson measure. 

The answer lies in the fact that the Swanson CPI does not take account of pervasive grade 
retentions in high school grades, particularly retention in the 9th grade (the 9th grade bulge). 
Swanson’s failure to deal with grade retention makes the CPI an unsatisfactory measure of either on- 
time or eventual graduation. Moreover, different states and school districts have different policies 
regarding grade retention. As a result, a naive application of the Swanson CPI without accounting 
for retention would confound differences in retention policy with differences in graduation rates and 
would be a unreliable measure to use for accountability purposes. 1 ’ 


10 The Swanson CPI is composed of four grade-to-grade progression ratios — one between the 9th 
and 10th grades, one between the 10th and 11th grades, one between the 11th and 12th grades, and the last 
one between the 12th grade and graduation. This synthetic product can be easily extended to the 8th grade by 
including the progression ratio between the 8th and 9th grades. 

11 The calculations in the first and second columns include 48 states. Enrollment data are missing for 
Washington DC in 2003—04. Diploma data for 2002—03 are missing from New Hampshire and South 
Carolina. 

12 All students who graduated within 5 years since the beginning of their 8th grade should be 
included in the on-time 8th-grade-to-graduation rate. However, some people who did not graduate in 5 years 
beginning in 8th grade but did graduate in 4 years beginning in 9th grade — e.g. 8th-grade repeaters who then 
completed high school in 4 years — will be included only in the on-time 9th-grade-to-graduation rate, thus 
making it greater than the on-time 8th-grade-to-graduation rate. 

13 Swanson acknowledges that grade retention is a potential problem. His response is that “the CPI 
graduation rate estimate, though not perfect, was the least susceptible to bias caused by the 9th grade 
enrollment bulge” (Orfield et ak, 2004, p. 10). He further adds that “However, it should be noted that an 
enrollment bulge caused the CPI and all other measures examined to underestimate , not overestimate, the 
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Greene et al. Another popular set of graduation rates has been published by Jay Greene and 
his coauthors (see Greene, 2002, Greene and Forster, 2003, Greene and Winters, 2005, Greene and 
Winters, 2006). Unlike Swanson, Greene and his co-authors calculate a final or eventual completion 
rate, rather than a four-year on-time rate. 14 Greene’s initial measure released in November 2001 
(revised in April 2002) compared diplomas to the eight grade enrollment four years earlier. As such, 
the resulting graduation rates are not distorted by retention rates during high school, including the 
ninth grade bulge. However, in later studies (Greene and Forster, 2003, Greene and Winters, 2005, 
Greene and Winters, 2006), Greene moved away from using 8th grade enrollment as the base. 
Greene and his coauthors currently acknowledge that retention of students at the 9th grade requires 
adjustments to the CCD enrollment count of 9th graders to produce an estimate of first-time 
( entering ) 9th graders. The two main components of the (current) Greene method are — first, 
averaging over the enrollments in the 8th, 9th and 10th grades (for a particular cohort) to estimate 
the size of the entering 9th grade for this cohort and second, inflating this estimate of entering 9th 
graders by growth in population during the cohort’s high school years to arrive at the final number 
for the projected graduating cohort — the number of students who could possibly graduate with this 
particular cohort or class. 

We show below that averaging the 8th, 9th and 10th grade enrollments for a particular 
cohort is unlikely to yield the correct estimate of entering 9th graders and that the bias is particularly 
large for minorities. Because this averaging is also advocated by recent NCES studies (e.g., Seastrom 
et al., 2005, 2007) and the draft regulations allowing states to use this type of measure as an interim 
substitute for longitudinal measures, it is important to explore this feature of Greene's research. The 
population adjustments in the Greene and Winters (2006) method can also be inaccurate. At the 
national level the net increase in the cohort size can only come from net international immigration 
during high school years, but we know little about who these people are, whether they enroll in U.S. 
schools after they immigrate, and whether educational attainment subsequent to enrollment reflects 
the performance of U.S. high schools or is more influenced by their educational experience in their 
native countries. Further, as shown in an appendix, due to a use of population estimates 
benchmarked to different census years, the population adjustments as reported in Greene and 
Winters (2006) are overstated and results in an underestimation of graduation rates, particularly for 
minorities. 

Seastrom et al. Researchers at the National Center for Education Statistics, an arm of the 
U.S. Department of Education, have borrowed Greene's averaging method and proposed the 
Averaged Freshman Graduation Rate (AFGR) (see Seastrom et al., 2005, 2007). This is similar to the 
Greene and Winters graduation rate in that it uses the average of 8th, 9th and 10th grade 
enrollments for a particular cohort as the best estimate for that cohort’s number of entering 9th 


actual graduation rate. Therefore, this suggests that all measures are currently overestimating graduation rates, 
and actual rates would likely prove even lower” (emphasis added). 

14 Greene mentions this issue in his paper, but there is still a popular perception that the Greene 
method yields a four-year completion rate. For example, the National School Board Association’s Center for 
Public Education notes in its website that “Many recent high-profile reports on high school graduation are 
based on four-year estimates, most notably, the Manhattan Institute’s methodology which calculates on-time 
graduation at about 70 percent (Greene, 2003)” (Center for Public Education, 2006; emphasis added). 
Similarly, Sara Mead at the Education Sector writes in a recent report, “Research by the Manhattan Institute 
found that only about 65 percent of boys who start high school graduate four years later , compared with 72 
percent of girls” (Mead, 2006, p. 10; emphasis added). 
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graders. The AFGR is obtained by dividing the number of diplomas issued in a year by this 
smoothed cohort enrollment. However, unlike Greene, the NCES studies do not have any 
additional population adjustments, so the AFGR graduation rates are higher. 13 

For example, for the 2002-03 school year, the averaged freshman graduation rate (AFGR) 
for public schools is 73.9%, obtained by dividing the number of public school diploma recipients for 
that school year (2,719,947) by the average of the 8th-grade public school enrollment for 1998-99 
(3,529,963), 9th-grade public school enrollment for 1999-2000 (3,986,992), and lOth-grade public 
school enrollment for 2000-01 (3,529,652) (Seastrom et al, 2007). For the 2003-04 school year, the 
AFGR for the 48 reporting states 16 and the District of Columbia, similarly calculated, is 75.0%. 

Gaps between 8th grade and averaged enrollments. Both the Greene studies and the NCES 
studies use the average of enrollments in 8th, 9th, and 10th grades as a proxy for the size of the 
entering 9th grade class. However, though this mitigates the 9th grade bulge problem in graduation 
rate calculations, averaging does not eliminate the problem. The averaged estimate of cohort size still 
falls short of the tme cohort size, the size of entering 9th graders. One can see the difficulty by 
comparing the national public 8th grade enrollment in one year to the averaged enrollments 
(following Greene et al. and Seastrom et al.) over that year and the following two years. The 
respective averaged enrollments for the classes of 2003 and 2004 were 3,682,202 and 3,396,916, 
more than 4% higher than the first year's 8th grade enrollment (3,529,963 and 3,261,969, 
respectively). There are only two ways that the national public school enrollment could grow 
between the 8th and 9th grades for a particular cohort — if there were a significant net in-migration 
at the national level or if there were a significant influx of people from private elementary and 
middle schools to public high schools. 1 Below we show that neither of these two factors can explain 
the increase in public school enrollment between the 8th and 9th grades as hypothesized by both the 
Greene and the NCES studies. 

Private schools and 9th grade enrollment. While it is tme that private schools serve 
proportionately more students at the primary level than the secondary level, the differences are not 
large in comparison to public school enrollment (see Table 6). Results from the 1999-2000 Private 
School Universe Survey conducted by the NCES show that 1999-2000 private school enrollment in 
8th and 9th grades was 369,579 and 336,224, respectively. 18 Even if the private-school enrollment 
difference is accounted for entirely by private-school students moving into public schools, such 
transfers would increase public 9th grade enrollment by less than 1% compared to previous year’s 
8th grade enrollment. 


15 The published NCES studies referred to above do not separately calculate graduation rates by race, 
though this is possible from the CCD data. 

16 The diploma counts for Wisconsin and New York were missing. 

17 Dropping out between the 8th and 9th grades will lead to an underestimate of the graduation rate, 
as compared to net influx of students in public schools at grade 9 which will lead to overestimates. Because 
we focus on the ways in which the graduation measures developed in recent studies are underestimates of the 
true graduation rate, we omit a discussion here of the issue of dropping out between 8th and 9th grades. 

18 See Broughman and Colaciello (2001), Table 11, page 15. These numbers include students in other 
than regular programs, e.g. students in special education and alternative programs, and those in a special 
program emphasis. Restricting analysis to regular elementary/secondary enrollment results in numbers of 
347,156 and 309,096, respectively. 
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Table 6 

Enrollment below college for people 3 to 24 j ears old, by control of school, sex, race, and Hispanic 
origin, 2004 


Elementary enrollment High school enrollment 
% Public % Private % Public % Private 


All Races 

90 

10 

92 

8 

Black, non-Hispanic 

95 

5 

97 

3 

Hispanic (of any race) 

95 

5 

96 

4 

White, non-Hispanic 

86 

14 

90 

10 


Source: U.S. Census Bureau, Current Population Survey, October 2004 


Given that 8th grade public school enrollment in 1998-99 was 3,529,963, the best estimate 
of 9th grade enrollment in 1999-2000 would be 3,529,963 plus transfers from private schools, plus 
net international immigration. The last one is most likely to be minimal at a national level — certainly 
less than 1% — and heavily concentrated among the Hispanic population. 19 Assuming that the entire 
decline in enrollment between the 8th and 9th grades in private schools is accounted for by transfers 
to public schools, this will imply that the best estimate of public 9th grade enrollment in 1 999-2000 
is 3,529,963 + (369,579-336,224) or 3,529,963 + 33,355 = 3,563,31 8. 20 The averaged freshman 
cohort size estimated in the NCES study for this cohort is 3.3% higher (3,682,202). If we divide the 
number of diplomas in 2002-03 (2,719,947) by our estimate of entering 9th graders, rather than the 
averaged cohort size as in the NCES study, the graduation rate increases to 76.3%. This is about 
10% higher than the two-thirds rate portrayed in recent studies including the Task Force of the 
National Governors’ Association (2005). 

How does that difference grow or shrink when looking at population subgroups? The NCES 
study does not break down graduation rates by race. However, Blacks and Hispanics are much less 
likely to attend private schools and much more likely to be retained in the 9th grade, so estimating 
the size of their entering classes by averaging over 8th, 9th, and 10th grade enrollments is likely to 
exaggerate the true cohort size more than for the general population. For the Class of 2003, a simple 
8th grade-to-diploma graduation rate (BCR-8; see Haney et al. 2004) is higher for Blacks and 
Hispanics than the AFGR (Seastrom et al., 2005, 2007). The differences are about 1.5% for non- 
Hispanic Whites but 5% for non- Hispanic Blacks and 7% for Hispanics. Separate calculations by 
race and ethnicity are important, because private school enrollment is notably smaller for Black and 
Hispanic populations compared to White. For example, private school attendance for Black students 
is 5% at the elementary level and 3% at the high school level. Even if we assume that the decline in 
enrollment at the high school level is entirely accounted for by net transfers to public schools, this 
will imply that the size of the entering 9th grade class for Black students at public schools is only 2% 
more than the 8th grade Black enrollment in public schools in the previous year. 21 For Hispanics, the 
increase is even lower, around 1 %. 


19 Calculations from the PUMS micro-data for 2000 American Community Survey show that about 
65% — or two-thirds — of those who immigrated to the U.S. at ages 13 and 14 were Hispanics. In contrast, the 
percentage of Hispanics among the native-born population was less than 10%. 

20 Since the Private School Universe Survey has only cross-section results for a particular year, the 
enrollment numbers in grades 8 and 9 used here — 369,579 and 336,224, respectively — refer to successive 
cohorts rather than the same cohort. However, it is unlikely that the size of the cohort would have changed 
significantly between these two years. 

21 Of all Black students in 8th grade, 95% are in public schools and 5% are in private schools. When 
this cohort moves on to 9th grade (high school), the 95% of students already in public schools are joined by 
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Further, most of the difference between the two measures for Whites can be explained in 
terms of net transfer from private schools to public schools between the 8th and 9th grades, but 
little of the gap for Black and Hispanic students can be explained in this way. Table 7 disaggregates 
enrollment and graduation estimates at the elementary and secondary level by school type for each 
race. The table calculates three different graduation measures: Seastrom et al.'s (2005, 2007) AFGR 
(using the average of the cohort year's 8th, 9th, and 10th grade enrollments), Haney et al.'s (2004) 
8th-grade-to-diploma measure (unadjusted for transfers or other population changes, as discussed 
below), and a ratio of diplomas to an adjusted 8th grade enrollment, assuming that the entire 
difference between 8th and 9th grade private-school enrollment moves into public schools as part of 
the cohort's entering 9th grade enrollment. 

The first three rows show the total enrollments in 8th, 9th, and 10th grades for the years in 
which this cohort would have been in these respective grades. The next row is the average of these 
three enrollments — the estimate of first-time or entering 9th graders used by Greene's team and 
NCES. The fifth row is the estimated percentage of students who could theoretically be net transfers 
from private to public schools, calculated using Census Bureau data in the same was as described for 
national data. The row below that is the theoretical maximum 9th grade entering cohort using 
private-to-public transfers. The next row is the number of diplomas reported for 2002-03. The last 
three rows calculate different graduate measures — the Adjusted Freshman Graduation Rate 
(Seastrom et al., 2005, 2007), the diploma-to-8th-grade-enrollment ratio (Haney et al., 2004), and the 
Haney measure after adjusting for the theoretical maximum of private to public transfers. 

Table 7 


Calculation of BCR-8, AFGR, and adjusted BCR-8 for the class of 2003 


Measure 

Whites 

Blacks 

Hispanics 

8th graders in 1998-99 

2,133,502 

515,071 

442,535 

9th graders in 1999-2000 

2,287,117 

649,807 

560,348 

10th graders in 2000-01 

2,103,388 

509,543 

463,360 

Average of 8th-10th graders 

Maximum net transfers from private to public 

schools between 8th and 9th grades (as % of 

2,174,669 

558,140 

488,748 

8th grade public school enrollment) 

8th grade public school enrollment augmented 

4.65% 

2.11% 

1.05% 

by complete net transfers 

2,232,735 

525,915 

447,193 

Diplomas in 2002—03 

1,727,896 

331,337 

311,977 

AFGR (Seastrom et al.) 

79% 

59% 

64% ~ 

Unadjusted 8th-to-diploma ratio (Haney et al.) 

81% 

64% 

70% 

Adjusted-8th-to-diploma ratio 

77% 

63% 

70% 


Source: Common Core of Data, NCES. This table includes 45 states and Washington DC. In 2003—04, 
these states accounted for more than 93% of total 8th grade enrollment in the country. (Enrollment data 
are missing for Arizona, Idaho and New Jersey. Diploma data for 2002—03 are missing from New 
Hampshire and South Carolina.) 

As one would expect from the lowest cohort estimate, the Haney measure yields the highest 
graduation rates. However, the ratio after adjusting for potential private-to-public transfers yields 


another 2% who (net) transfer from private to public schools. The increase in the public school cohort size 
between the 8th and 9th grades is about 2%. 
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rates very close to the unadjusted ratio for Blacks and Hispanics. The AFGR, on the other hand, 
gives significantly lower graduation rates — a 5% difference for Black students and a 7% difference 
for Hispanics. While international migration might affect Hispanic measures, it is unlikely to affect 
the same measure for Black students. Since the main motivation for using an averaged measure 
instead of 8th grade enrollment is the transfer of students from private to public schools, this 
analysis suggests that the averaged enrollment is susceptible to differential bias by population 
subgroup. The divergence of the measures strongly suggests that averaging 8th, 9th, and 10th grade 
enrollments is a misleading proxy for the number of entering 9th graders. 

In other words, the maximum net transfers from private to public schools increases the size 
of the entering 9th grade class by 1-2% at best for Black and Hispanic students. On the other hand, 
averaging the 8th, 9th, and 10th grade enrollments gives estimates of entering 9th grade class which 
exceed the corresponding 8th grade class by more than 8% higher for Black students and more than 
10% higher for Hispanics. Since net international immigration is likely to be much less than 1%, 
averaging 8th, 9th and 10th grade enrollments is an inappropriate proxy for the size of the entering 
9th graders — it leads to graduation rates which are biased downwards, particularly so for Black and 
Hispanic students. Using what is arguably the best proxy for the size of the entering 9th grade 
class — 8th grade enrollment in public schools in the previous year augmented by estimates of net 
transfers from public to private schools, the CCD-based enrollment and diploma numbers yield a 
graduation rate of about 77-80% for whites, 63-65% for blacks and 69-71% for Hispanics. Note 
that the graduation rates for the minorities are 1 5—20% higher than the commonly-cited 50% figure. 

8th-grade-to-diploma rates 

Haney et al. Other researchers in the field have proposed their own measures. One of the 
simplest measures has been proposed by Haney et al (2004) — this is the total number of diplomas 
issued in year t+5 divided by the number of 8th graders in year t. In other words, this is a 8th-grade- 
to-diploma graduation rate, without any additional adjustments that Greene or NCES proposes. 
Perhaps not surprisingly, this 8th-grade-to-diploma measure — called Basic Completion Rate or 
BCR-8 by Warren (2005) — is often the one closest to graduation rate estimates from other sources. 22 

Warren. One of the most recent methods of calculating graduation rates is the Estimated 
Completion Rate or ECR by Warren (2005). The two defining characteristics of the ECR are its use 
of 8th grade enrollment as the best predictor of next year’s entering 9th graders, and its adjustment 
of the size of the entering 9th graders by a migration adjustment similar to Greene’s. Based on actual 
retention and grade-repetition data from Massachusetts, North Carolina, and Texas, Warren argues 
that the best predictor of entering ninth graders in public schools in year /+ / is the number of 
eighth graders in public schools in year A 2 ’ He adjusts this number for migration by comparing “the 
total population of 17-year-olds — the modal age of fall twelfth graders — in a state on July 1 of one 


22 Miao and Haney (2004) surveyed the most important measures of graduation used at the time and 
concluded that there was “no evidence that the conceptually more complex methods yield more accurate or 
valid graduation rate estimates than the simpler methods.” 

23 Warren has a brief discussion of the potential sources of bias from using 8th grade enrollment 
without any further adjustments. In particular, he notes that grade retention in the 8th grade might impart a 
downward bias while transfers from private schools to public schools between 8th and 9th grades might 
impart an upward bias to the estimates. However, in findings similar to ours, he argues that analysis of 2000 
census data indicates that in only 9 U.S. states did the percentage of 5th— 8th graders attending private schools 
differ from the percentage of 9th— 12th graders attending private school by as much as 2%, suggesting that the 
second effect is likely to be small. 
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year to the total population of 13-year-olds — the modal age of fall eighth graders — in that state on 
July 1 four years earlier.” The ECR shows a graduation rate of 71.9% in 2000, 71.1% in 2001 and 
72.2% in 2002. 24 

As mentioned earlier, a net increase in the cohort size at the national level can only come 
from net international immigration during high school years. However, we know little about who 
these people are, whether they at all enroll in U.S. schools after they immigrate, and if they do enroll, 
whether their subsequent educational attainment reflects the performance of U.S. high schools or is 
more influenced by their educational experience in their native countries. In calculations using 
census micro-data (PUMS data) from the 2000 decennial census, we found that recent immigrants 
coming to the U.S. have much lower levels of educational attainment, and it is unlikely that most of 
them ever attend U.S. high schools. As a result of differential attainment, adjustments based on the 
population of 17 year olds are problematic, as it will include many of these recent immigrants. 2 ’ 

Warren (2005) argues that the ECR is validated by the fact that the ECR comes close to the 
on-time graduation rate obtained from the NELS. For the class of 1992 the NELS gives an on-time 
completion rate of about 79.6% for public school students. 26 The ECR, on the other hand, shows a 
graduation rate of 78.4% without the migration adjustment. However, this comparison is incorrect 
for at least two reasons. First, to make the ECR comparable to the NELS, one has to exclude not 
only the migrants from the denominator, as Warren does, but also the number of diplomas going to 
the migrants from the numerator. We do not think there is a way of estimating teen migrants' 
diplomas from the published CCD statistics. If one only takes out migrants from the denominator 
but not the diplomas going to the migrants from the numerator, one overestimates graduation rate. 2 
Second, the NELS graduation rate (79.6%) to which Warren compares ECR is based on a 
comparison of the number of entering 9th graders to the number of diplomas awarded three years 
hence to this cohort. But the appropriate comparison is to the 83% graduation rate the NELS 
reports for 2000, when diplomas awarded in the years past normal senior year are included. -6 This 
more closely matches the ECR which is calculated to include all diplomas awarded in a particular 


24 Warren does not compute the ECR for specific racial or ethnic groups. 

25 Most net immigration into the U.S. is concentrated among people in their late teens and early 
twenties. Warren admits that his measure counts international in-migrants who come to the U.S. between 
ages 13 and 17 — but never enroll in high school — as non-completers, but argues that this exerts only modest 
downward bias on the ECR, most notably in states with high levels of international in-migration. 

26 Different researchers come to slightly different figures for the on-time graduation rate of the NELS 
cohort. Warren’s own calculations (for public school students only) put it at 79.6%. Adelman (2006, Table 
LI) argues that it is 78.3%. Heckman and LaFontaine (2007) argues that it is 77.3%. 

27 Our calculations using the 2000 census micro-data (IPUMS), not reported, suggest that not many 
of these in-migrants will complete high school in the U.S. It is unlikely that many of them will even attend 
one. So the extent of bias may not be large. However, given that Warren’s migration adjustment for this 
cohort is about 5.6% — implying that the size of this cohort increased by 5.6% between the 8th and 12th 
grades — this may still bias the calculated graduation rate upwards to some extent. 

28 Studies by the NCES (Adelman, 2006; National Center for Education Statistics, 2004) show that 
83% of 1988 8th graders in the NELS had completed high school with a regular diploma by 2000, though 
Heckman and LaFontaine (2007) argue that that actual figure is about 79.7%. These numbers are higher than 
Warren’s estimate of 78.4%, which itself is possibly slightly overestimated due to inclusion of diplomas 
obtained by immigrants who came to the country after age 13 (or 8th grade). The NELS figure includes both 
public and private school students, but because private schools enroll less than 1 0% of all high school 
students the bias on that account is likely to be minimal, less than 1%. In addition, as Warren points out (page 
18), the ECR comes closest to the NELS figure — the CPI is 71.2% in 1992, an apparent underestimate of 
about 9—12%. 
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year relative to the number of entering 9th graders three years ago. 24 Since in any given year there 
will be some diplomas awarded to students who have taken more than four years to complete, the 
ECR is not an on-time graduation rate. 

Recent Recalculations 

Thus far, we have discussed proposed measures that use one main data stream. However, 
the use of multiple sources is important. In our earlier work (Mishel & Roy), we compared the 
various graduation rates (Swanson, Greene, Haney, and Warren) computed with school enrollment 
data to the results from three longitudinal studies which are based on the same school-based data 
but track individual students. Our earlier work compared these school-based rates to those of a 
study that tracks Chicago students and shows graduation rates from 1996 to 2004. From these 
comparisons we argued that the conventional measures not only understate graduation rates but that 
they are also mistaken in establishing trends: the longitudinal data shows steady progress (up to 8%) 
but the conventional measures show no progress. We also compared graduation rates based on 
longitudinal data of New York City students against the conventional measures. We find that the 
new conventional measures yield graduation rates that are 10% lower than the longitudinal 
measures. Finally, we compared the more popular published measures with the longitudinal rates 
published by the state of Florida using the cohorts graduating in 2002 and 2003. These graduation 
rates are at least ten to 15% higher than the graduation rates produced by the conventional school 
enrollment-based measures. ’" That is, the computations that underlie the new conventional wisdom 
are seriously inaccurate. While there can be minor quibbles about the exact definitions of graduation 
used by these different places and the reliability of the data used, we concluded in part from these 
comparisons that the recendy-proposed CCD-based measures are poor proxies for tracking 
graduation rates, falling significantly short of replicating the underlying picture. Two important 
recent papers reanalyzing graduation rates and attempting to reconcile different measures. We 
discuss them below. 

Warren and Halpern-Manners (2007) 

In a recent paper, Warren and Halpern-Manners (2007) address the discrepancy in estimates 
of graduation rates calculated from the Current Population Survey (CPS) and CCD. One of the 
motivations for this is that in earlier work (Mishel & Roy, 2006), we had pointed out that CCD- 
based estimates (with adjustments) yield much lower graduation rates than most other sources, 
including the CPS. Warren and Halpern-Manners use the status drop-out rates for 16-24 year olds 


29 The exact formula for the ECR is 

ECR = High School Completers Springof AcademicYearx 

Estimated#of First - time 9th graders FallofAcademicYearx . 3 * Migration Adjustment 

30 In terms of individual measures, the 9th-grade-to-diploma and the CPI perform the worst, with 
more than 15% differences for all groups except Hispanics, for which the difference is still around 10%. Of 
the three recent measures proposed, Warren’s ECR does the best, but even here the differences are in general 
around 10% except for non-Hispanic Blacks. The simple 8th-grade-to-diploma measure is the one closest to 
the five-year cohort-graduation rate (but still about 3.0% too low) but seems an inappropriate proxy since it 
significantly overstates the Hispanic graduation rate (by about 7.5%) and understates the non-Hispanic white 
rate (by about 9.0%). For the detailed comparisons, see Table B-l from Mishel and Roy (2006). 
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from the CPS to eventually construct a measure of completion rate similar to those calculated from 
CCD data. They argue that most of the difference between their CPS-based measure and the CCD 
graduation rate is due to the fact that some CPS respondents report that their children are enrolled 
in (the latter years of) high school when they are not enrolled at all. They provide some suggestive 
evidence from the NELS longitudinal survey that sometimes parents whose children have dropped 
out of high school still report that their children are enrolled. 

It is important to note that the CPS measure that Warren and Halpern-Manners use is 
somewhat different from the one we used in our 2006 study (Mishel & Roy, 2006). Since the CCD 
based estimates yield a completion rate rather than a dropout rate, we compared the former with 
estimates of high school completion from the CPS. The particular measure we used is the educational 
attainment of the 25-29 year old population — one’s high school education is essentially over by the 
time he or she is 25 years old — which yield a final completion rate, comparable with those from the 
CCD. Jl Note that the CPS measure of completion that we use is similar to the one used by 
Heckman and LaFontaine (2007), who use educational attainment (high school completion) data for 
both the 20-24 and 25-29 age groups, and from both the March and October CPS supplements, 
restricting the sample to those who ever attended school. 

Second, while Warren and Halpern-Manners make a valid point that parents’ misstatements 
about their children’s enrollment status might bias CPS dropout rates downward, the CPS measure 
we use — high school completion status of current 25-29 year olds — is unlikely to be biased by 
misstatements about “being at school.” As discussed below, Heckman and LaFontaine (2007) agree 
with Warren and Halpern-Manners that the discrepancy between CPS and CCD-based graduation 
rates is mostly concentrated in one category (students who enroll in 12 th grade, but drop out before 
getting a diploma), but they argue that this is more due to the particular survey design in CPS rather 
than willful misreporting by parents. In particular, they argue that the above discrepancy is due to 
proxy response bias. 32 In personal communication, Heckman and LaFontaine write that “though 
sometimes the proxies are unable to make fine distinctions (such as 12th grade, no diploma vs. HS 
graduate) in general they do get it right”. They further argue that “as far as misreporting [is 
concerned], it is clear from NELS and NLSY data that people do respond honestly to questions. 

The number of discrepancies between self-reports and administrative transcript reports are small.” 

So it is likely that misreporting regarding high school completion status in the CPS, when the question 
is directed at people aged 25-29 years old who have been out of high school for a while, will be 
small and the bias minimal. 

Because they are not directly comparable, dropout data from the CPS can not be matched as 
such against graduation data from the CPS. A closer measure is the one about high school 
completion status of people at a certain age that both we and Heckman and LaFontaine (2007) use. 
After appropriate adjustments, CPS-based estimates of graduation are much closer to CCD-based 
estimates, as shown by both Mishel and Roy (2006) and Heckman and LaFontaine, though some 
differences remain, particularly for minorities. 


31 There are other issues of comparability — the CPS surveys include all students, whether they 
attended public or private schools, whereas CCD data only pertain to public school students. Second, the 
CPS surveys consider GED recipients as high school completers,- we have to net out the number of GED 
recipients from total completers to get an estimate of regular diploma holders, comparable to CCD. 

32 Proxy response bias refers to the fact that in the CPS surveys, unlike many other surveys like the 
Census or the American Community Survey (ACS), one person responds for the entire household. Heckman 
and LaFontaine note that this discrepancy is not present in estimates from the Census and the ACS. 
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In a recent paper, Heckman and LaFontaine (2007) pursue the same issues we did in earlier 
work (Mishel & Roy, 2006), with a methodological approach similar to ours. In that work, we 
showed that the conventional wisdom about high school graduation is seriously inaccurate. We had 
analyzed data from every important source we could identify, including CCD data which form the 
main basis for the current conventional wisdom, longitudinal surveys which follow individual 
students over time and individual level micro-data from the 2000 census and the Current Population 
Survey. In all cases we developed estimates of the graduation rate with regular diplomas, excluding 
GEDs. Heckman and LaFontaine they take this approach one step further. They provide estimates 
of graduation rates from earlier censuses (1970, 1980 and 1990), add a few additional surveys which 
provide estimates of high school graduation (the NLS68 and Add Health), and attempt to reconcile 
the divergence in earlier studies. 

Heckman and LaFontaine (2007) agree with our harsh assessment of the high profile 
computations of high school graduation rates using CCD data. In particular, they agree with our 
consequences about the large and growing 9th grade bulge — the fact that an increasing number of 
students are being retained in the 9th grade (that is, these students repeat grade 9). They agree that 
the bulge makes any estimates of graduation rates problematic when the measure relies on 9th grade 
enrollment from the CCD. They concur with us and with Walt Haney that the best estimate of 
graduation rate using CCD data is obtained by comparing the number of diplomas with the number 
of students in 8th grade 5 years earlier — the 8th-grade-to-graduation rate. Heckman and LaFontaine 
note that other “widely-used estimators that condition on 9th grade enrollment greatly 
underestimate graduation rates” and that “estimated minority graduation rates miss the mark 
completely ” (p. 18; emphasis added). None of the recent studies which claim that minorities have 
only a 50% chance of graduating from high school use the 8th-grade-to-graduation measure as their 
graduation rate. There are potential problems with using 8th-grade-to-graduation ratio, such as the 
transfer of students from private schools to public schools, but our assessment has been that such 
biases are very small on a national level. 33 Note, however, that as shown in Mishel and Roy (2006), 
even this 8th-grade-to-graduation measure sometimes yields graduation rates for minorities very 
different than those from other sources/’ 


Conclusion 

Understanding the level and trends in high school graduation in the U.S. is cmcial to a 
proper understanding of educational attainment in the country and the challenges we face as we 
prepare our workforce for the 21 st century. A number of recent studies have asserted that only two- 
thirds of all students in the U.S. graduate — and that blacks and Hispanics have only a 50% chance of 
graduating with a regular high school diploma. This claim is based on grade-specific enrollment and 


33 Since private-to-public transfers may differ geographically, however, this is worth considering for 
comparison in specific circumstances. 

34 The major difference between Mishel and Roy (2006) and Heckman and LaFontaine (2007) is in 
delineation of the long-run trends in high graduation. While we argue in our book that there have been 
significant improvements in high school graduation over the last 30—40 years, particularly for minorities 
(though the progress stalled for black men in the 1990s), Heckman and LaFontaine argue that high school 
graduation has been declining over the past 40 years and that the racial gaps in graduation have not 
converged. 
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diploma data published by the U.S. Department of Education. However, because these enrollment 
and diploma numbers do not yield a direct graduation rate, researchers have used different 
adjustments to construct one using these data. We show in this paper that these adjustments often 
impart significant downward bias to the measures of graduation rates and portray an incorrect 
picture of true high school graduation. 

In previous research (Mishel & Roy, 2006) we examined a range of data sources which 
contain information on high school graduation, including the decennial census, household surveys, 
longitudinal surveys which track individual students as they progress through high school (and verify 
their progress against actual transcripts). Based on the most reliable data sources, we find that more 
than 80% of high school students graduate with a regular diploma and that the graduation rates for 
minorities lie somewhere between two-thirds and three-fourths — about 70%. Furthermore, of the 
one-quarter of black students who drop out, about half eventually earn a GED (general education 
development), which qualifies them for entry into post-secondary institutions and the military. We 
also find that there has a significant progress over the last 25 years in improving minority graduation 
rates and closing the Black- White and the Hispanic-White gaps in high school completion. 

The importance of a high school diploma cannot be underestimated. However, we want to 
create a better understanding of the tme challenges we face and help lead the way to better targeted 
solutions for continuing to close the remaining gaps. There are significant problems to be 
addressed — the minority graduation rates are still low, and there are significant gaps in completion 
between whites and Asians on the one hand and blacks and Hispanics on the other. In some inner 
cities such as Chicago, Black males have only a 40% chance of completing high school with a regular 
diploma. However, we believe that unless we know the true picture we are unlikely to correctly 
address these problems. The new graduation rate measures that use CCD data along with other 
adjustments can often yield misleading estimates and should be used with caution. 
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Appendix A 

Graduation Rate Estimates from Recent Studies 


Table A-l 

U.S. high school graduation rates with a regular diploma, recent high school classes 


Study 

Grad 

Year 

Graduation Rates with a Regular DipL 
All Black Hispanic Asian 

oma 

White 

Greene (2001) 

1998 

71.0 

56.0 

54.0 

— 

78.0 

Greene & Forster (2003) 

2001 

70.0 

51.0 

52.0 

79.0 

72.0 

Swanson (2004) 

2001 

68.0 

50.2 

53.2 

76.8 

74.9 

Greene & Winters (2005) 

2002 

71.0 

56.0 

52.0 

— 

78.0 

Warren (2005) 

2002 

72.2 

— 

— 

— 

— 

Greene & Winters (2006) 

2003 

70.0 

55.0 

53.0 

72.0 

78.0 

Education Week (2006) 

2003 

69.6 

51.6 

55.6 

77.0 

76.2 

Education Week (2008) 

2004 

69.9 

— 

— 

— 

— 


Both the Education Week studies use the method proposed by Chris Swanson (see Swanson 2003, 2004) 
called the Cumulative Promotion Index or CPI. 
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Appendix B 

Are the population adjustments in Greene and Winters (2006) 

reasonable and accurate? 

Greene and Winters (2006) argue that because the size of a cohort changes from the time 
this cohort enters the 9th grade to the time the students belonging to this cohort are in the 12 th 
grade, the calculated graduation rates should take account of this. They inflate their estimate of 
entering 9th graders by the increase in the population during these three years, proxying the increase 
by the difference between the number of 17 years old in 2002 and the number of 14 years old in 
1999. 

Because these years span 2000, the year of the decennial census, these estimates of the 
number of 14 year olds in 1999 and 17 year olds in 2000 are not directly comparable. The estimates 
of 14 years olds in 1999 are benchmarked to the 1990 census, while the estimates of 17 year olds in 
2002 are benchmarked to the 2000 census. This creates a problem — mostly because in the 2000 
census the Census Bureau found many more people than it expected to find — as is easily seen in the 
following table. 

Table B-l 

Differences in population cohort growth from 14 to 15, due to change in benchmarking from 1990 
census to 2000 census 


% growth, cohort size between ages 14 and 15 


Year that cohort was aged 14 

Total 

White 

Black 

Hispanic 

Asian 

1995 

1.2 

0.4 

2.3 

3.4 

2.7 

1996 

1.3 

0.5 

2.4 

3.5 

2.6 

1997 

1.2 

0.5 

2.3 

3.2 

2.3 

1998 

1.2 

0.4 

2.3 

3.3 

2.5 

1999 

5.8 

1.9 

7.4 

15.5 

17.7 

2000 

0.5 

0.1 

0.3 

1.9 

2.4 

2001 

0.5 

0.1 

0.3 

1.8 

2.3 


Source: Authors’ calculations from the Census Bureau and data provided by Jay Greene. 


This table shows the discrepancy between benchmarking to the 1990 census and 
benchmarking to the 2000 census, as evidenced by the experience of the cohort aged 14 years in 
1999. The 14-year old population for this cohort was benchmarked to the 1990 census, while the 15- 
year old population is benchmarked to the 2000 census. While the average change in cohort size 
between ages 14 and 15 is close to 1% for all earlier and later cohorts, for the cohort aged 14 years 
old in 1999 this jumps up to 6% — primarily because for this cohort the number of 14 years old and 
the number of 15 years old are coming from different sources. (For all the other cohorts, the 
numbers come from the same basic source and are comparable.) The bias is much larger for the 
minorities, particularly the Hispanics and the Asians, though it is also significant for the Blacks. This 
spike in population adjustment significantly affects the calculation of graduation rates, as the 
following table shows. 
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Table B-2 

Effect of population adjustments on Greene' s graduation rates, class of 2003 

Population Changes Graduation Rates Difference 


Group 

Change 

(Greene) 

Net of 
2000 spike 

Unad- 

justed 

Adjusted 

(Greene) 

Adjusted 
(Roy & 
Mishel) 

Greene 

Roy & 
Mishel 

White 

2% 

0% 

79% 

78% 

79% 

-2% 

0% 

Black 

8% 

1% 

59% 

55% 

58% 

-4% 

-1% 

Hispanic 

21% 

6% 

64% 

53% 

60% 

-11% 

-4% 

Asian 

25% 

7% 

90% 

72% 

84% 

-18% 

-6% 

All 

7% 

2% 

75% 

70% 

73% 

-5% 

-1% 


In this table we assume that the true population change between 1999 and 2000 for the 
cohort aged 14 years old in 1999 is best approximated by taking the average of the corresponding 
changes for the previous cohort (cohort aged 14 years old in 1998) and the succeeding cohort 
(cohort aged 14 years old in 2000). Under this reasonable assumption, the graduation rates calculated 
by Greene’s method increases to 73% overall, and to 60% for Hispanics and 58% for Blacks. ' 6 The 
graduation rate for Asians jumps to 84% and is higher than that of Whites at 79%. 

An alternative adjustment for population growth — where we assume that the population 
change between 1999 and 2000 for the cohort aged 14 years old in 1999 is close to the average for 
this cohort between ages 15 and 16 (2000 to 2001) and between ages 16 and 17 (2001 to 2002) — 
gives similar but slightly higher graduation rates. The overall graduation rate is 74% by this method, 
with the Black rate at 59%. 


36 See Table B-l for these rates of growth. Note that for Hispanics and Blacks in particular the rates 
of growth in cohort size were somewhat smaller in the post-1999 period, though the differences are small. 
(The rates for Asians are mostly unchanged.) 
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